Add grouped order by operator #997

kevin-dp · 2025-12-10T10:51:48Z

Follow up on #993 which introduces a grouped topK operator. This PR introduces a grouped orderBy operator.

This commit introduces a new operator that allows for topK operations to be performed independently on groups of data. Co-authored-by: kevin.de.porre <[email protected]>

Introduce TopKState<K, T> to encapsulate the common state and operations for managing a single topK window (multiplicity tracking and topK data structure). Both TopKWithFractionalIndexOperator (single instance) and GroupedTopKWithFractionalIndexOperator (one instance per group) now use this helper class, eliminating code duplication. Also extract handleMoveIn and handleMoveOut as standalone functions that can be reused by both operators.

This commit introduces the `groupedOrderBy` operator, which allows for ordering and limiting elements within distinct groups. It also includes necessary exports and tests for this new functionality. Co-authored-by: kevin.de.porre <[email protected]>

cursor · 2025-12-10T10:51:49Z

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

changeset-bot · 2025-12-10T10:51:51Z

🦋 Changeset detected

Latest commit: e9305b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 13 packages

Name	Type
@tanstack/db-ivm	Patch
@tanstack/db	Patch
@tanstack/angular-db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/offline-transactions	Patch
@tanstack/powersync-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

pkg-pr-new · 2025-12-10T10:53:36Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@997

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@997

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@997

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@997

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@997

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@997

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@997

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@997

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@997

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@997

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@997

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@997

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@997

commit: e9305b8

github-actions · 2025-12-10T10:56:38Z

Size Change: 0 B

Total Size: 88.2 kB

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/change-events.js`	1.39 kB
`./packages/db/dist/esm/collection/changes.js`	977 B
`./packages/db/dist/esm/collection/events.js`	388 B
`./packages/db/dist/esm/collection/index.js`	3.24 kB
`./packages/db/dist/esm/collection/indexes.js`	1.1 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.67 kB
`./packages/db/dist/esm/collection/mutations.js`	2.31 kB
`./packages/db/dist/esm/collection/state.js`	3.42 kB
`./packages/db/dist/esm/collection/subscription.js`	2.71 kB
`./packages/db/dist/esm/collection/sync.js`	2.37 kB
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	4.19 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/index.js`	2.66 kB
`./packages/db/dist/esm/indexes/auto-index.js`	742 B
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.93 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.1 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	513 B
`./packages/db/dist/esm/local-only.js`	837 B
`./packages/db/dist/esm/local-storage.js`	2.1 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.75 kB
`./packages/db/dist/esm/query/builder/functions.js`	733 B
`./packages/db/dist/esm/query/builder/index.js`	3.96 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	917 B
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.35 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	430 B
`./packages/db/dist/esm/query/compiler/group-by.js`	1.8 kB
`./packages/db/dist/esm/query/compiler/index.js`	1.96 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.46 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.07 kB
`./packages/db/dist/esm/query/expression-helpers.js`	1.43 kB
`./packages/db/dist/esm/query/ir.js`	673 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-config-builder.js`	5.33 kB
`./packages/db/dist/esm/query/live/collection-registry.js`	264 B
`./packages/db/dist/esm/query/live/collection-subscriber.js`	1.74 kB
`./packages/db/dist/esm/query/live/internal.js`	130 B
`./packages/db/dist/esm/query/optimizer.js`	2.56 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.91 kB
`./packages/db/dist/esm/query/subset-dedupe.js`	921 B
`./packages/db/dist/esm/scheduler.js`	1.3 kB
`./packages/db/dist/esm/SortedMap.js`	1.3 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	247 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	428 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	246 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	881 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	852 B
`./packages/db/dist/esm/utils/cursor.js`	457 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.51 kB
`./packages/db/dist/esm/utils/type-guards.js`	157 B

_{compressed-size-action::db-package-size}

github-actions · 2025-12-10T10:57:14Z

Size Change: 0 B

Total Size: 3.35 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	225 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.17 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.12 kB
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	431 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

KyleAMathews

Code Review: PR #997

Overall Assessment: Approve ✅

A well-structured addition that builds on the foundation laid by PR #993. Like the Doctrine and Covenants building upon the Book of Mormon, this PR extends the grouped topK operator to provide a more ergonomic grouped ordering interface.

Summary

The PR introduces a groupedOrderByWithFractionalIndex operator that wraps groupedTopKWithFractionalIndex with consolidate() to provide grouped ordering functionality.

Strengths

Thin wrapper design: The implementation is elegantly simple - it delegates to the existing groupedTopKWithFractionalIndex and adds consolidation:
```
return stream.pipe(
  groupedTopKWithFractionalIndex(...),
  consolidate(),
)
```
Comprehensive test coverage: The test file covers:
- Array key grouping
- Value-based grouping via groupKeyFn
- Incremental updates within groups
- Removal from topK (and backfill)
- Multiple independent groups
- Offset support
- Custom comparators
- Delimiter-based key extraction
- Infinite limit (no limit)
- setSizeCallback integration
- Moving window with setWindowFn
- Empty group cleanup
- String property ordering
Good TypeScript generics: The type inference for KeyType and ValueType is well handled.

Minor Notes

groupedOrderBy.ts:73-75 - The cast chain is a bit complex:
```
type StreamKey = KeyType extends string | number ? KeyType : string | number
```
This is reasonable given the type constraints, but worth a comment explaining why this is needed.
Dependency on PR #993: Since this builds on groupedTopKWithFractionalIndex, ensure #993 is merged first.
Default comparator: The default comparator uses JS comparison operators:
```
if (a < b) return -1
return 1
```
This works well for primitives but note that for objects, it will use [object Object] string comparison. Consider adding a note in the JSDoc.

Question

Is there a use case for exposing a simpler groupedOrderBy (without fractional indexing) for scenarios where fractional indices aren't needed? Or is the fractional index always useful for maintaining stable ordering?

Excellent extension to the operator toolkit! 🎯

samwillis

Also LGTM

cursoragent and others added 9 commits December 10, 2025 10:20

feat: Add groupedTopKWithFractionalIndex operator

e0d072e

This commit introduces a new operator that allows for topK operations to be performed independently on groups of data. Co-authored-by: kevin.de.porre <[email protected]>

ci: apply automated fixes

506cc8d

ci: apply automated fixes

6193beb

Refactoring of the topK operator and grouped topK operator

88dbcf6

ci: apply automated fixes

fef17b6

Improve unit tests

f4e1cd6

ci: apply automated fixes

6e03bd8

feat: Add groupedOrderBy operator

fdd892b

This commit introduces the `groupedOrderBy` operator, which allows for ordering and limiting elements within distinct groups. It also includes necessary exports and tests for this new functionality. Co-authored-by: kevin.de.porre <[email protected]>

ci: apply automated fixes

276332e

kevin-dp and others added 5 commits December 10, 2025 13:35

Improve unit tests

e12c558

Remove unnecessary changes

3910674

ci: apply automated fixes

6065818

Fix failing test

d17bf7b

ci: apply automated fixes

e9305b8

kevin-dp marked this pull request as ready for review December 10, 2025 12:58

kevin-dp requested a review from samwillis December 10, 2025 12:59

kevin-dp added this to TanStack DB 1.0.0 release Jan 6, 2026

kevin-dp moved this to Ready for review in TanStack DB 1.0.0 release Jan 6, 2026

KyleAMathews reviewed Jan 8, 2026

View reviewed changes

Base automatically changed from cursor/grouped-top-k-operator-34bd to main January 12, 2026 13:44

samwillis approved these changes Jan 12, 2026

View reviewed changes

kevin-dp merged commit f5b504e into main Jan 13, 2026
7 checks passed

kevin-dp deleted the cursor/add-grouped-order-by-operator-d343 branch January 13, 2026 12:47

github-actions bot mentioned this pull request Jan 13, 2026

ci: Version Packages #1123

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add grouped order by operator #997

Add grouped order by operator #997

Uh oh!

kevin-dp commented Dec 10, 2025 •

edited

Loading

Uh oh!

cursor bot commented Dec 10, 2025

Uh oh!

changeset-bot bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

pkg-pr-new bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

KyleAMathews left a comment

Uh oh!

samwillis left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Add grouped order by operator #997

Add grouped order by operator #997

Uh oh!

Conversation

kevin-dp commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor bot commented Dec 10, 2025

Uh oh!

changeset-bot bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KyleAMathews left a comment

Choose a reason for hiding this comment

Code Review: PR #997

Summary

Strengths

Minor Notes

Question

Uh oh!

samwillis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kevin-dp commented Dec 10, 2025 •

edited

Loading

changeset-bot bot commented Dec 10, 2025 •

edited

Loading

pkg-pr-new bot commented Dec 10, 2025 •

edited

Loading

github-actions bot commented Dec 10, 2025 •

edited

Loading

github-actions bot commented Dec 10, 2025 •

edited

Loading